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Abstract 

People who claim that applying Novakian concept maps in Turkish is problematic base their arguments 
largely upon the structural differences between the English and Turkish languages. This study aims to 
introduce the indexing method to eliminate problems encountered in Turkish applications of Novakian 
maps and to share the preliminary results of its application. For this purpose, 20 pre-service chemistry 
teachers who had mastered the concept-mapping method after seven weeks of intense education were asked 
to prepare concept maps on the topic of atoms by using both the Novakian and indexing methods. The maps 
were qualitatively assessed by experts regarding informational content. The participants were asked to state 
in writing their opinions about the indexing method. The resulting analysis revealed that the participants 
were able to express the relation between two concepts more realistically when they used the indexing 
method, and the resulting maps were much richer in informational content. Almost all participants (95%) 
stated that the new method was more convenient in terms of reflecting the necessary mental information. 
The results of the study show the indexing method to be a much more convenient and informative way of 
preparing concept maps for both teachers and students. 
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The meaning of a concept depends on how it is understood in relation to other 
concepts. In other words, in order to understand what a concept means, one must 
see how it is placed into an informational structure (Taber, 2002, p. 31). Mastering a 
certain topic means having sound knowledge about the related conceptual structure 
(Ruiz-Primo, Shavelson, & Schultz, 1997a). Concept maps can enlighten a significant 
portion of these relations (Ruiz-Primo, Schultz, & Shavelson, 1997b). This is why 
concept maps are very useful in promoting meaningful learning and determining 
whether this type of learning has taken place or not. The various uses of concept 
maps in the learning process are as follows. Concept maps: 

• help students understand how a concept is mentally located during the learning 
process (Novak, 1990). 

• provide teachers with the opportunity to organize knowledge in the educational 
process and offer students the opportunity to find key points and major principles 
in the teaching material (Novak, 1998, p. 27). 

• are very useful for teachers in perceiving students’ mental models and in 
planning lectures in accordance with their needs and interests (Kinchin, Hay, 
& Adams, 2000). 

• indicate inadequacies in students’ informational structure (Francisco, Nakhleh, 
Nurrenbem, & Miller, 2002). 

• are unique graphical tools for showing how students should organize, relate, 
and synthesize knowledge. This is why concept maps are highly useful for both 
teachers and students (Vanides, Yin, Tomita, & Ruiz-Primo, 2005). 

• can be used to determine how students relate concepts, their prior knowledge, 
and their alternative conceptions (Novak & Gowin, 1984; Ruiz-Primo & 
Shavelson, 1996). 

• are very useful for enlightening students’ misconceptions (Ta§<;i, 2015; Djanettea 
& Fouadb, 2014; Aykutlu & Sen, 2012; 1 1 dlr & Sen, 2006; Karamustafaoglu, 
Ayas, & Co§tu, 2002). 

Concept mapping is a method developed by Novak in the 1970s based on Ausubel’s 
(1968) meaningful learning theory (Novak & Gowin, 1984; Novak & Musonda, 
1991). Concept maps are two-dimensional schemes that enable one to express 
concepts mentally and relate them pictorially (Ruiz-Primo, Schultz, Li, & Shavelson, 
2001; Safayeni, Derbentseva, & Canas, 2005). Its basic units are concepts and 
relationships among them. Concepts are connected to each other through relational 
lines. The relationship (a word or short phrase) is written above these lines (Novak & 
Canas, 2008). The resultant phrase expressing the relationship between two concepts 
is called a proposition, the major component of concept maps (Novak & Gowin, 
1984; Ruiz-Primo et al., 2001). 
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Concept maps were first developed by Novak in English. In accordance with 
the syntax structure of the English language, if one follows the order of the first 
concept, relational phrase, and second concept, one is provided with a meaningful 
clause describing the relationship between the concepts. This is because the syntax 
order of the English language (subject + verb + object) is highly suitable for this 
type of application. Elowever, in non-Latin, agglutinative languages such as Turkish, 
Elungarian, Mongolian, Finnish, Slovakian, Korean, or Japanese where the syntax 
order differs being a non-modal suffix-additive type, the application of this method 
becomes highly problematic because the sequential reading of 1 st concept + verb + 2 nd 
concept do not often provide a meaningful clause. Lee (1999) claimed that the syntax 
difference in Korean and Japanese makes the preparation of Novakian concept maps 
highly cumbersome. Park and Kim (2008) stated that there are syntax problems in 
preparing concept maps in Korean, and they added an adverbial word between the 
concepts to avoid these types of problems. 

The syntax order of the Turkish language (subject + object + verb) causes 
problems if concept maps are prepared based upon English syntax. If one reads a 
content map prepared in Turkish in the sequential order of the English language, the 
resulting clauses are often meaningless or hard to understand. In order to obtain a 
meaningful clause in Novakian concept maps in Turkish, it must be read in the order 
of 1 st concept + verb + 2 nd concept. Sometimes the order of 1 st concept + 2 nd concept 
+ verb gives almost meaningful phrases. Examining the examples of propositions in 
Figure 1, if one chooses the first option in the first proposition, a perfectly meaningful 
clause is obtained, such as “ element atomlardan olu$ur ” (Elements are formed from 
atoms) in total compliance with Turkish syntax. If the second order in the second 
proposition is chosen, then one ends up with the understandable but somewhat 
distorted clause “ atomlar kimyasal yollarla birle§erek olu§tururlar bile§ik” (Atoms 
combined through chemical ways form compounds). The change of the reading order 
according to the language used causes quite a bit of ambivalence when concept maps 
are used for evaluative purposes (note: If one literally translates from Turkish into 
English, the clause may make sense in English but may be hardly understandable and 
distorted according to Turkish syntax. Therefore some sentences will be given in total 
accordance with the reading order in order to emphasize this point). 
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Figure 1. Examples of propositions. 

The fact that English is a prefixed language and Turkish is suffixed, along with 
the difficulty in expressing suffixes in concept maps, causes inevitable shifts in 
expressions. In Turkish, in order to form an understandable and syntaxially sound 
clause, one has to add suffixes to the verb (inge?, 2009). When examining the first 
proposition in Figure 1, it seems to mean that elements are formed from a single 
atom due to the difficulty involved in writing the suffixes into the concept maps. If 
this expression had been written in answer to an open-ended question, it would be 
regarded as a misconception. Therefore it is highly difficult to determine whether the 
misconception was intrinsic to the student or the student had written it like that due 
to the nature of concept maps. The evaluation must be free of these sorts of dilemmas 
to be reliable and valid. 

This immense difference between English and Turkish lingual syntax causes 
problems in preparing Novakian concept maps in Turkish (Turan & Ekmekci, 2011; 
inge?, 2009; Saglam, 2009; Unlii, Inge?, & Ta§ar, 2006; Bagci K 1 I 15 , 2003). Unlu 
et al. (2006) reported in their study that they had had too many problems due to the 
syntaxial differences between Turkish and English and that the expressions between 
the concepts describing their relationship were often not compatible with Turkish 
grammar. They claimed that this syntaxial distortion may well result in misconceptions. 
Similarly, Sozbilir and Neac§u (2014) stated that the incompatibility of Novakian 
concepts maps with Turkish had caused problems in describing relationships between 
concepts. They said that these problems were the greatest disadvantage in using 
concept maps in Turkish. Didi§, Ozcan, and Azar (2014), in their study carried out 
with physics teachers working in schools where the medium of instruction was in 
Turkish or English, discovered that participants had too many problems constructing 
concept maps in Turkish, which was further verified by analyzing the resultant 
concept maps, inge? (2009) reported in her study on pre-service physics teachers that 
the participants’ concept map scores were much lower than their scores on multiple- 
choice questions. The researcher concluded that they knew the concept but could not 
express them on their maps. This situation was attributed to the syntaxial difference 
between Turkish and English. Turan and Ekmekci (2011) in their study involving 22 
pre-service chemistry teachers regarding concept maps, determined that one of the 
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problems that had immediately emerged was the syntaxial incompatibility between 
Turkish and English. Similarly, Unlii et al. (2006) observed that participants had 
managed to perfectly relate the concepts with each other, but had failed to write 
down the relational clauses. The authors explained this situation as being due to the 
syntaxial difference between Turkish and English, pointing to the poorly constructed 
clauses that resulted from phrases employed to describe the relationship between 
two concepts. In the study carried out by Bilici, Dogan, and Avci (2015) where they 
investigated the effectiveness of concept maps in Turkish as an education tool most 
students were found to have a hard time writing relational phrases between concepts. 
Bagci K 1 I 19 (2003), in his study carried out upon 134 primary education students, 
realized that 63.6% of students had difficulty writing relational clauses between 
concepts and that they had resorted to alternative methods, such as writing a complete 
sentence above the concepts, writing the relationship in textual form under the map, or 
trying to make oral explanations. Although all three methods remedied the problems 
stemming from writing the relationship in a one- or two-word phrase in Turkish, 
they had certain limitations (Saglam, 2009). Writing whole sentences complicated 
the maps and made them extremely difficult to evaluate. The second method did not 
allow the reviewer to make a relation-based assessment. The third method made the 
map evaluations take too long. 

The first method designed to obviate the problems involved in Novakian maps in 
Turkish was Saglam’s (2009) numbering method. According to this method, relational 
phrases are replaced by numbers, which are explained in complete sentences on a 
different page due to inadequate space on the concept-map page. The author found 
that, although this method resulted in uniform concept maps, it was difficult to follow 
the relationships and explanations on a separate page (Saglam, 2009). The method 
was observed to correct the poorly constructed sentences for the propositions, but it 
demanded more time to write explanations onto a separate page, which made it very 
cumbersome to follow. It also violated the basic principle of concept maps, which is 
to have all concepts and relationships seen on the same page. 

According to Novak and Gowin (1984), concept maps can be employed as a 
learning strategy, a tool for planning to teach curriculum, and an evaluation tool. 
Similarly, Taber (2002) claims that concept maps can be utilized as educational, 
evaluative, learning, and self-assessment tools. The readability of concept maps 
becomes much more important when used as an assessment tool for illuminating 
how students relate concepts to each other (Edmonson, 2000, p 20). Assessors can 
only measure participants’ mental concepts as reflected on paper. Therefore, the 
reliability and validity of assessments are largely dependent upon the clarity of 
phrases on the maps. People who claim that preparing Novakian maps in Turkish is 
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highly problematic point to the problem of reading maps in Turkish as evidence of 
their claim (Bagci K 1 I 19 , 2003; Didis et al., 2014; Saglam, 2009; Turan & Ekmekci, 
2011; Unlii et al., 2006). 

The main goal of concept maps, different from mind-maps, is to elucidate concepts 
and the relations among them to fully illuminate the conceptual structure (Novak & 
Musonda, 1991). Therefore, writing relationships clearly and understandably is of 
double importance with regard to elucidating students’ conceptual structures through 
the use of concept maps (Novak & Gowin, 1984). §ahin (2001) found that pre-service 
teachers used concept maps mostly to see concepts in an organized manner. Similarly, 
Giindiiz (2014) found that teachers resort to concept maps for presenting topics in 
an integrated, meaningful, and permanent format. If one thinks in this pretext, the 
problems encountered in preparing concept maps in Turkish are the greatest barrier 
to using them in line with this purpose. Bagci K 1 I 15 (2003) claims that it is not 
possible to write in Turkish the relationships among concepts in concept maps that 
were originally designed for English due to the syntaxial differences between these 
two languages. Because of that, the author claims that Novakian concept maps are 
not suitable for Turkish. However, concept mapping is a very unique method for 
enlightening conceptual structures in students’ minds, and is too valuable to abandon 
it due to some application problems. At this point, we suggest a new method, the 
indexing method, developed by the authors to increase the readability of concept 
maps and to provide the opportunity to express concepts more distinctly and clearly. 
The purpose of this study is to illustrate the efficiency of the indexing method and 
gather pre-service teachers’ opinions about using this method. 

Method 


Pilot Study 

This study is part of the first author’s doctoral dissertation, investigating and 
comparing three types of existing concept maps: Novakian, numbered, and fill-in-the- 
blank. All three methods were assessed by a group of pre-service chemistry teachers in 
terms of their advantages and disadvantages. The participants (N = 25) were observed 
to have difficulties, particularly with finding relations in Novakian concept maps, 
which are supposed to provide grammatically correct clauses when read sequentially. 
The participants stated they had failed to write the appropriate phrase even when 
they knew the relation. This situation was subjected to in-depth analysis to define and 
find a remedy to these problems. The authors developed an alternative method, the 
indexing method, which could be a solution to all these problems. 
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Developing the Indexing Method 

The indexing method was developed over the subject of covalent bonds with 
five participants chosen arbitrarily from the entire sample of the pilot study. The 
participants first prepared Novakian concept maps, which were assessed with regard 
to the relationships among concepts. The participants were then interviewed by the 
author and asked to verbally state the relations on their concept maps. The participants 
had been observed to verbally explain 50% of the relations correctly, but this ratio 
was not reflected onto their concept maps. They were then asked what should be 
done to remedy this situation. The relative problem was determined to be certain 
situations where the relational phrase between two concepts should include a third 
concept as well. For example the true relationship between atoms and covalent bonds 
should be “a covalent bond is formed between nonmetallic atoms” ( kovalent baglar 
ametal atomlan arasinda olu§ur). However, almost every participant expressed this 
as “the covalent bond is formed between atoms” (kovalent bag atomlar arasmda 
olu§ur). They realized that something was missing in this picture and immediately 
stated that “it is not between every atom but between the nonmetallic atoms” during 
the interview. It was as if the problem could be solved by separating the relational 
phrase. However, another problem immediately emerged: What should be the order 
of words when the phrase is read? There author proposed an ingenious way to cope 
with this problem: The relational phrase is separated into two parts and indexed 
with a “1” and “2;” these numbers refer to Concept 1 and Concept 2, as numbered 
arrows also indicate. If the relational phrase indexed like that is read as Concept 1 
+ first-indexed proposition followed by Concept 2 + second-indexed proposition, 
it gives a perfectly meaningful clause in complete compliance with Turkish syntax. 
The participants were then asked to construct the relation in accordance with the new 
method. All five participants unanimously agreed that the new method was better 
than Novakian concept maps for expressing their knowledge. The effectiveness of 
the indexing method was thoroughly compared with Novakian concept maps in the 
framework of the main study. 

Indexing Method 

This method provides a new point of view on Novakian maps and is thought to 
be able to simplify and increase their usefulness. The basic principle of the new 
method is to construct meaningful sentences from the relational phrases by using the 
indexing process. If the indexed proposition formed between two concepts is read in 
the order of 1 st concept + first-indexed relational phrase then 2 nd concept + second- 
indexed relational phrase, one ends up with a perfectly sound sentence describing 
the relationship. For instance, the proposition between the concepts of atom and 
ion (see Figure 2) when not using the indexing method result in the clause “atom 
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(atom) electron (electron) kaybederek (by losing) veya (or) kazanarak (by gaining) 
olu§turur (forms) iyon(ion),” which, though sensible in English, is incompatible 
with the syntaxial structure of Turkish. On the other hand, when using indexing, this 
proposition becomes atomlar elektron kaybederek veya kazanarak iyon olu§tururlar, 
this literally translates as atoms electron by losing or by gaining ion form', this 
complies perfectly with Turkish grammatical structure. In addition to the suffix added 
at the end of verb to complete the sentence in Turkish, a (+) can be written in front of 
the relational phrase to connect the numbered concepts appropriately. 

t -lar electron kaybederek 
veya kazanarak 
-s by losing or gaining 
electron 



1st concept 2nd concept 

Figure 2. Indexing method. 

The conceptual inadequacy in Turkish for the proposition from Figure 1 ( element 
atomdan oluyur, or element from atom is formed) is eliminated when it is subjected 
to indexing method ( elementler aym tiir atomlardan oluyur, or elements from the 
same types of atoms are formed). It is clear that the use of indexing results in a much 
more conceptually sound and meaningful Turkish clause. The fact that suffixes to be 
used are indicated on the same page as the concepts makes the resulting clause much 
more meaningful. 


1 -ler aym tiir 
(s) the same type 

2 -lardan olujur 
are formedfrom 

Figure 3. An example of a proposition prepared using the indexing method. 

The indexing method developed by the authors is based upon Novak’s method. 
Placing a single word upon the arrow between concepts is enough to indicate the 
relationship between some concepts. For instance, if one takes the concepts of 
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elements and atomic number and places the word ordered above the relational arrow, 
the final clause would be given as elements are ordered according to their atomic 
numbers. The indexing method is better suited for expressing relations that require 
more than one word. For one-word simple expressions, the customary Novakian 
approach fully suffices. 


Research Design 

The case-study research design was used here in order to make an in-depth analysis 
of the participants’ concept maps, which they had prepared using two different 
approaches, and to verify the effectiveness of the indexing method. Case-study 
research models provide in-depth information about a situation by using qualitative 
and quantitative techniques (Patton, 2002). It is mainly employed to unveil details of 
a complex situation and collect in-depth information on it (Taber, 2007). 

Participants 

The study was carried on 20 female and three male pre-service teachers studying in 
their 4 th year at the Chemistry Education Department. Flowever, the male participants 
were discarded from the study because they had contributed very little to the whole 
study. C a talk ay a (2005) stated that gender plays no role in the process of making 
concept maps. Therefore, the fact that the participants were formed only from female 
students causes no problem. The study was carried out using the purposive sampling 
method, which is in complete accordance with normal qualitative studies. According 
to Patton (1987), random sampling allows one to make valid generalizations due 
to its representational power, while purposive sampling enables one to perform in- 
depth analyses and draw detailed information about the case (Yildirim & Simijek, 
2008). The participants had seven weeks of detailed education on constructing and 
grading concept maps. During this period, the participants prepared concept maps 
using three different methods (indexing, Novakian, numbering) on different topics in 
chemistry (matter, periodic table of elements, covalent bonds). Also, their education 
was directed toward equipping the participants with the capacity to evaluate concept 
maps. The purpose of this education was to have the pre-service teachers equipped 
with expert knowledge about preparing and grading different types of concept maps 
so as to increase the reliability of the study. Srinivasan, McElvany, Shay, Shavelson, 
and West (2008) stated that mastery increases with constant learning, practice, 
experience, and feedback. After seven weeks of intense training, the participants 
were regarded to have become sufficiently acquainted with preparing and assessing 
concept maps. 
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Data Collection and Analysis 

The effectiveness of the newly developed indexing method was verified in three 
different ways, as listed below: 

1. The content of the propositions obtained through the indexing method were 
compared with similar propositions on Novakian concept maps. 

2. The maps were converted into a textual format by their peers, and these final 
texts were examined by the author. 

3. Content analysis was performed using the pre-service teachers’ opinions. 

In order to test the effectiveness of the indexing method, the participants were 
asked to prepare concept maps with 14 concepts about atoms using the Novakian 
and indexing methods. The purpose of this practice was to define whether there were 
any differences regarding content as a result of the mapping method, regardless of 
using the same concepts. The author examined the relationships fonned by each 
participant in both maps one by one using content analysis in terms of the conceptual 
correctness and syntaxial clarity of the established propositions. There were 570 
propositions stored on computer media. The relationships between the same concepts 
established by each participant in two different maps were compared with regard 
to clarity and correctness. Phrases that showed differences regarding conceptual 
content and correctness on the different maps were indicated in different colored 
boxes. The relationships were classified under five categories: correct phrase, wrong 
phrase, conceptual error, inappropriate for the scientific model, and phrase with 
excess information. The analysis table of one participant has been given in Figure 
5 as an example. The operational descriptions of the categories are presented in 
the Appendix. The analysis tables of each participant were checked by an expert 
in regard to the coding of each phrase. Inappropriately coded relations were re¬ 
coded. Reliability between coders was computed using the formula of [agreement 
/ (agreement + disagreement)] x 100, as developed by Miles and Hubennan (1994). 
The reliability coefficient was found to be 95.08 after the expert’s examination. There 
were a total of 570 propositions from both Novakian and indexed concept mappings 
(285 from each type). There were incompatibilities in 28 of the propositions between 
coders. These differences were eliminated after nine phrases observed to contain 
conceptual errors and mislabeled as “correct” were re-coded as “conceptual error,” 
seven phrases coded as “concept error” were re-coded as “incompatible with the 
scientific model,” and 12 phrases coded as “correct” were re-coded as “ phrase with 
excess information.” 
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Explanations 

Novakian proposition 

Indexing proposition 

Explanation 

Correct phrase 

Elementler “-dan olu$ur” atomlar. 

(elements ‘-formedfrom " atoms) 

Elementler farkli cins atomlar dan olu$ur. 

(element are formed by same apes of atoms) 

Phrase with excessive information 

Correct phrase 

Atomlar “haline domi$ebilir” iyon. 

(Atoms "converted” ions) 

Atomlar elektron ali§ verifi yaparlarsa iyon durumuna 
ge9er. 

(Atoms are converted in to ions when they exchange 
electronsj 

Phrase with excessive information 

Incompatible with the scientific 

Yoninge “-da yer alir" atomlar 
(Orbital "-is in" atoms) 

Atomlarda yoninge bulunur. 

(orbitals are in atoms) 

Incompatible with the scientific 

Wrong phrasethe arrow is 
placed m wrong direction 

Atom modeli “-m tarihsel geli$iminde yer alir” 
atomlar. 

(atomic model "are in the historical development of' 
atoms) 

Atomlar tammlanirken 4 fe§it atom modelinden 
yararlamlir. 

(There are 4 types of atomic models used in the 
description of atoms) 

Correct phrase 

Correct phrase 

Qekirdek “-da yer alir" atomlar. 

(Nucleus "-are in" atoms) 

Atomlarda fekirdek bulunur. 

(There is a nucleus in an atom) 

Correct phrase 

Incompatible with the scientific 

Elektron ‘"-de yer alir” yoninge. 

(Electrons "are in" orbital) 

Yoriingede hareket halinde elektronlar bulunur. 

(There are moving electrons in orbital) 

Incompatible with the scientific 

Correct phrase 

iyonik bilejik “arasinda elektron akjveri^iyle olujur" 

atomlar. 

(Ionic compound "are formed by electron 
exchangebetween" atoms) 

iyonik bilegiklerin formtilleri iyon yuklerinin 

(japrazlanmasiyla olu^turulur. 

(Ionic compounds formulas are formed by crossing 
ionic charge) 

Phrase with excessive information 

Correct phrase 

Anyon “jefididir” iyon. 

(Anion "is a tvpe of' ion) 

iyon elektron aldiginda anyon olu§ur. 

(When ion gains electron an anion occurs) 

Concept error 

Correct phrase 

Katyon “<;e§ididir" iyon. 

(Cation "is a tvpe of' ion) 

iyon elektron verdiginde katyon olu$ur. 

(When ion loses electron an cation occurs) 

Concept error 

Correct phrase 

Notron “-te yer alir” ^ekirdek. 

(Neutron "are in" nucleus) 

Cekirdekteki yuksuz taneriklere notron denir 
( The uncharged particles called neutrons in the 

Phrase with excessive information 

Correct phrase 

Proton “-te yer alir” jekirdek 
(Proton "are in" nucleus) 

Cekirdekte bulunan - yiiklii taneciklere proton dewr 
(The positive charged particles called protons in the 
nucleus) 

Phrase with excessive information 

Correct phrase 

Atom numarasi “sayisi olarak da soylenir" proton 
(Atomic number "is also called as " number of 
protons) 

Atom numarasina fekirdek yukii de denir. 

(The atomic number is also called as nuclear charge) 

Correct phrase 

Correct phrase 

Ktitle numarasi na baglidir” atom numarasi 
(mass number “depends on the" atomic number) 

Atom numarasi notron sayisi ile toplanmca ktitle 
numarasi elde edilir. 

(when number of neutron and atomic number summed 
up mass number is obtained) 

Phrase with excessive information 

Concept error 

izotop atom “farklidir" atom numarasi 
(isotope atoms "are different" atomic numbers) 

izotop atomlarda atom numarasi farklidir. 

(The atomic numbers of isotope atoms are different ) 

Concept error 


Figure 4. Content analysis of Novakian and indexed concept maps. 


The pre-service teachers were then asked to write down their opinions about the 
two methods, taking into account the maps they had prepared. They carried out this 
activity within the framework of the questions, “Which method do you think was more 
successful, why, and what point do you base your conclusion upon?” The opinions 
of the participants were subjected to a thorough content analysis. The basic process 
of the content analysis is to gather similar concepts and themes and arrange them in 
wording that everyone can understand (Yildirim & Simijek, 2008). The pre-service 
teachers’ clauses were examined by the primary researcher who had established the 
codes and raw themes. Forty-two codes and six basic themes were established for 
the participants’ phrases. The raw data was first coded by the main author, then the 
data was subjected to a thorough review by two experts for the reliability of the 
research. The first expert was given the raw data (not including the coding carried out 
by the researcher) and was asked to code from the beginning. For this purpose, both 
the operational descriptions of the themes were prepared and an expert examination 
form that included the operational descriptions was formed. The forms were given to 
an expert with a PhD in a qualified field, and she was asked to carry out the coding 
again. The coding of the researcher and the expert were compared, and consistency 
of the coding was determined by labeling each code as “agreed” or “disagreed.” 
The reliability between coders was determined by using the following formula: 
[agreements / (agreements + disagreements)] x 100, as developed by Miles and 
Fluberman (1994). The fact that the coding process was carried out by an expert 
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who had started from scratch increased the reliability of the study. The expert was 
allowed to use new theme names. When the codes and theme names determined by 
the researcher and the second coder were examined, one disagreement regarding the 
name of a theme remained between them. The reliability ratio was found to be 80.95 
between the two coders. Taking this low value into account, the researcher, together 
with the first expert who had carried out the coding process, re-examined all the 
codes and theme names. The resulting codes and theme names were given to a second 
expert for the final check. In other words, the researcher and the first expert carried 
out the coding separately, and the second expert checked both of them. The codes and 
themes, which had taken their final shape, were presented with excerpts taken from 
the participants’ opinions. 

The maps that the pre-service teachers prepared using two different methods 
(Novak and Indexing methods) were photocopied while hiding the names of the 
participants and arbitrarily distributed to the other participants. The reason for 
distributing the different types of maps to various peers was to obviate drawing 
alternative conclusions from the second map in case the first map was not understood. 
The participant, acting as a peer, was asked to give a number to each relation and 
write down the relation as she understood it. The purpose of this practice was to 
determine whether the message of the maps was clear to someone other than the 
participants and whether the preparation method had caused any difference in terms 
of meaning when the map was read by a peer. Figure 5 shows an example of the maps’ 
peer review. The maps were then distributed to 40% of the participants, arbitrarily 
chosen in order to have different maps read by different peers. The goal here was to 
determine whether two different peers would read the propositions in the Novakian 
and indexed maps the same way. The maps that the participants had converted into 
a text format were examined by the researcher last. The practice of having the maps 
read by another person serves the purpose of understanding how participants formed 
the relations between two concepts. This concluded our quest to compare Novakian 
and indexed maps regarding readability. The data were presented together with the 
participants’ propositions. 


2122 



Turan Oluk, Ekmekci / A Different Approach to Preparing Novakian Concept Maps: The Indexing Method 



J. AKTf'lar.O^OI" "'OdflltA! lit QOVIOAr 

A+«rr> rnodcMeyi Bokr a*o<r>oe*Fli otob>*ir 
3 _ 3u<l*r(*d cnon> nrvxW^. oloC'-».> 

<4 Aroivs a*o>i ~'0<J«*eA aUb>/' 

5. A4c* n macklfcW, &OMo«* oKwv. /Aodali olebW>>. 

6 . A-tw>lar, c-«ku-d*k *C/erf/". 

? - **«Mor ekiL**w> f 

g. &le/v«cAtW. oiOfY\lorti«»' oH-cur 

3 _ A tv'r'l«^ / d«J>Kjrob ,,; rt«^ 

(0 - kxj-Kjort ■ '^trckir 
/(. Afvjefv lyondur 

1 2 ' AA^SA, k»+^n joo.'t trltfi’k cVi^furvirleT 

13 ttomeMIe/. clemcA* >«e 

III. *G»le ««*TOT®»1, «!«''"•* jog Gii g&i*yil' r 

(5- il««MA+ ifrt'baWi''^ 4*1 • gA*te/*li^ 

'< iJcWy oteff>,o*«m -.j^orc»i ot^Mdir 
'1- ihetop o*X>ro tu*k rumo-a), -totkltdi/ 

I? - A-*0«V> 1'vmoroi 1 pro ■*■■>-. joju,*. lex 

19 - iroiop o+arA o£-4n>r\ sO^lon ^<v Lht£f 

JO - Protoo, n&lrOA pek."r«<#t It OukjriLlf 

21 - fcl«t*ron &«tird*t eifofindo A*r«fc£* Celts 


Figure 5. An example of a peer-read concept map prepared in accordance with Novakian concept 
maps. 

Reliability and Validity 

The data-triangulation strategy was employed in order to establish the study’s 
validity. Data triangulation is based upon collecting data using different data- 
collection tools. The consistency of data collected by different tools indicates that 
validity had been established (Guion, 2002). The effectiveness of the indexing 
method’s development was further verified in three different ways: content analysis 
of the concept maps, content analysis of the pre-service teachers’ opinions about the 
method, and content analysis of the texts that the peers had converted from the maps. 
All of these contributed to the validity of the method. 

Consistency between coders was established in order to verify validity of the 
data obtained from the pre-service teachers’ opinions about the effectiveness of the 
method and the data from the content analysis of the concept maps. The consistency 
of the data obtained from the participants’ opinions was found to be 80.95. The 
corresponding value for data obtained from the concept maps’ content analysis 
was 95.08. The consistency values of the data from the participants’ opinions were 
supported by their statements, and the data obtained from the concept maps’ content 
analysis were supported by the propositions stated in the maps. 
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Findings 

This part is devoted to the content analysis of the participants’ Novakian and 
indexed concept maps and to determine the effectiveness of the indexing method. 

Data Obtained from Content Analysis of the Concept Maps and Peer Reading 

The data obtained from the maps and peer-read texts regarding the content of the 
maps support each other and are provided under the same topic. 

Investigating the Novakian and indexed maps that the participants had prepared 
on the same topic in terms of content shows poorly constructed relational clauses 
to be much more prevalent in Novakian maps due to the unclear order in which 
propositions are read. The inherent difficulty of including informational content also 
has a big effect on this problem. Additionally, examining peer-read texts shows the 
Novakian texts to not be uniform and to have been read differently by different peers. 
Peers generally read the propositions incorrectly in Novakian mapping. 



(b) 


Figure 6. Mehtap’s concept maps. 
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Figure 6 contains the (a) Novakian and (b) indexed concept maps prepared by 
the same participant. When examining the Novakian map, it is unclear whether the 


propositions are to be read in the order of 1 st concept + relation + 2 nd concept or 1 st 


concept + 2 nd concept + relation. For instance, when the proposition between the 
concepts of elementler (elements) and atomlar (atoms) is read in the 1 st concept + 
2 nd concept + relation order, a perfectly meaningful clause is formed ( elementler 
atomlardan olu$ur, or in English, elements are fonned from atoms). When the 
proposition between the concepts of ivon (ion) and atomlar (atoms) are read in the 
1 st concept + relation + 2 nd concept order, one obtains the clause atomlar electron 
kaybederek veya kazanarak oluytururlar iyon (atoms by losing or gaining electrons 
they form ion), which is relatively understandable but grammatically distorted. In 
the exemplified sample, the propositions in Novakian maps were found to be read 
differently (with different meanings) by two participants. The first peer read the 
proposition as “atomlar iyon elektron kaybederek veya kazanarak iyon olu§tururlar” 
(the atoms and ions by losing or gaining electrons fonn ions) and the second peer 
read the proposition as “Atomlar elektron kaybederek veya kazanarak olu§tururlar 
iyon” (atoms by losing or gaining electrons form ion). This shows that the order is 
unclear and that the reader has to take the initiative in establishing a logical clause. 
This decreases the reliability of the method. On the other hand, the relationship gives 
a perfectly logical and grammatically correct clause with the indexing method if one 
obeys the general order of 1 st concept + Indexed Phrase 1 + 2 nd concept + Indexed 
Phrase 2. If one takes the same concepts of iyon and atomlar, the result is a very 
suitable clause: “Atomlar electron kaybederek veya kazanarak iyon olu§tururlar” 
(Atoms fonn ions by losing or gaining electrons). The way to read the final proposition 
does not depend on the assessor and is immutable. 





(•) 


Atom 

Modelled 



Atomlar 



Figure 7. A section of Esma’s concept maps. 
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(a) 0>) 

Figure 8. Melis’s concept maps. 


When examining the concept maps in Figures 7 and 8, the content is seen to be 
much richer and more explanatory in the indexed maps (b) than the Novakian maps 
(a). For instance, the grammatically incorrect and hardly understandable clause 
“Atomlar yapilarmi apiklar atom modelleri” (It explains the structures of atoms the 
atomic models) between the concepts of atoms and atomic models in Novakian maps 
becomes perfectly syntaxial and understandable in the indexed method: “Atomlarm 
yapisi atom modelleri ile a^iklamr” (The structure of atoms is explained by atomic 
models). Because the reading order is obscure in Novakian maps, the proposition was 
read as “Atomlar atom modelleri yapilarmi amildar” (Atoms explain the structure of 
atomic models), which was regarded as an incorrect relationship and graded as zero 
by a peer. However, the fact that the same peer read the same proposition correctly in 
the indexed map indicates that the ambivalence directly stems from the difference in 
how the two mapping methods are used. Also, the participants could only write short 
explanatory phrases such as §eklindedir (shaped), iqerir (contains) and oluqturur 
(forms) when using Novakian maps. However, when they used the indexing method, 
they could write a detailed phrase explaining the correct relationship between 
concepts. 

Investigating the Novakian and indexed concept maps revealed that the relationships 
established between some concepts showed marked differences. The explanatory 
clauses formed using these two methods were observed to be much more meaningful 
and syntaxially correct with the indexing method. The participants’ relational phrases 
between the concepts of atom and ion with the Novakian and indexing method are 
presented in Table 1. 
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Table 1 

Examples of Propositions Established by Participants Using the Novakian and Indexing Methods 

Participant 

Proposition in Novakian concept maps 

Proposition in indexed concept maps 

Melek 

Atomlar “haline d6nii§ebilir” iyon. 

(Atoms “can turn into ” ions) 

Atomlar elektron ali§ veri§i yaparlarsa iyon 
durumuna geger 

(If atoms transfer an electron, they become 
an ion) 

Sinem 

Elementler “-lardan olu§ur” atomlar 
(Elements “occur from ” atoms) 

Elementler aym cins atomlardan olu§ur. 
(Elements consist of the same atoms) 

Beyhan 

Atomlar “olu§turabilirler” iyon. 

(Atoms “can form ” ion) 

Atomlar elektron kaybederek veya kazanarak 
iyon olu§tururlar. 

(Atoms, by losing or gaining electrons, form 
ions) 

Fatma 

Atomlar “olu§turabilirler” iyon 
(Atoms “can form ” ion) 

Atomlar elektron ahp vererek iyon olu§tururlar 
(Atoms form ions by taking or giving elec¬ 
trons) 

Zeynep 

Atom “haline gegerken electron verir ya da 
ahr” iyon (atom “takes or gives an electron 
while becoming” ion) 

Atomlar electron vererek ya da alarak iyon 
haline gegebilir 

(atoms may become ions by taking or giving 
electrons) 



Figure 9. Sections from participants Defne’s (a) and ipek’s (b) concept maps. 


When the concept maps (Figure 9) prepared by participant “Defne” is examined, the 
Novakian maps result in some poorly worded and grammatically incorrect relational 
clauses because of the limitation on suffixes. When using the concepts of elements 
and atoms, the proposition obtained by the use of Novakian mapping was “element 
tek tip i?erir atomlar (element single-type includes atoms). The participant wants 
to express the relation “elementler tek tiir atomlardan olu§ur” (elements are formed 
from one kind of atom), but she couldn’t due to Novakian concept map restrictions. 
The clause was read as “elements atoms single” in the peer reading process, which 
was syntaxially incorrect and did not convey correct information to the reader. On the 
other hand, indexed mapping solved this problem, and the information was conveyed 
in both a syntaxially compatible and scientifically correct manner as “elementler tek 
tip atomlardan olu§ur” (elements are formed from single types of atoms). This relation 
was read correctly in the peer reading. Similarly, the relational clauses obtained by 
participant “ipek,” which investigated the relationship between atoms and nucleus 
using Novakian and indexing methods, were “?ekirdek merkezinde atom” (atom in 
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the center of the nucleus) and “atomun merkezinde gekirdek bulunur” (nucleus is 
found in the atom’s center). The absence of suffixes and the incompatible order of 
the clause elements resulted in virtually meaningless clauses in Novakian maps. The 
indexing approach seems to be a perfect remedy for all these problems. 



1 pv-tom rxj/vO'O-i' 



Figure 10. A section of the (a) Novakian and (b) indexed concept maps from pre-service teacher 
“Kevser.” 


The final proposition of Novakian maps in Figure 10 is highly distorted, 
grammatically incorrect, and confusing because the suffixes that should be added at 
the end of the verb are not allowed on the maps. The indexing approach is a solution 
to these types of problems and gives more grammatically and conceptually correct 
clauses. The relation between atoms and elements is “atomlar bir araya gelerek 
olu§turur element” (atoms together by combining forms element), obtained using 
Novakian concept maps. This relation reads much more clearly and grammatically 
correct as “aym tiir atomlar bir araya gelerek elementleri olusturur” (the same type of 
atoms come together to form elements) in the indexed approach. Not only did the peer 
correct the grammatical format, she also added a new informational phrase. When all 
these facts are taken into account, one can easily conclude that the grammatical and 
conceptual flaws in the final proposition were not due to a lack of knowledge but to 
the intrinsic nature of the mapping system. 
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Figure 11. The Novakian and indexed concept maps drawn by participant “Sinem.” 
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Figure 12. The Novakian and indexed concept maps drawn by participant “Melek.” 

Figure 11 shows the concept maps using the Novakian and indexing methods 
prepared by the same participant, “Sinem.” The indexing method seems to be much 
richer in terms of conceptual content. When examining the relationship between the 
isotope atom and mass number, the Novakian method ended up with the proposition 
“izotop atom kittle numarasi farkli olan atomlardir” (isotope atomic mass number 
are atoms that are different) while the proposition for the indexing method was 
“izotop atomlarm kittle numarasi birbirinden farkhdtr” (Isotopes have a different 
mass number than the atom’s mass number). The first proposition may result in the 
misconception that every atom with a different mass number is an isotope of one 
another. The second proposition poses no risk of that misconception. The indexing 
method clearly provides a much wider possibility of explaining complicated systems. 
This situation is also apparent in the relation between cations and ions. The Novakian 
approach uses small one- or two-word phrases, whereas the indexing system uses 
more explanatory phrases and suffixes as are used in Turkish (see Table 2). 
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Table 2 

Propositions Obtained from the Novakian and Indexing Methods 


Participants The propositions in Novakian maps 


The propositions in the indexed maps 


Beyhan 


Katyon dur” iyon / Anyon 6 — dur” iyon 
(Cation “is an ” ion / Anion “is an ” ion) 


Katyon pozitif yiiklii iyondur / Anyon negatif 
yiiklii iyondur 

(Cation is a positively charged ion / Anion is 
a negatively charged ion) 


Deniz 


Katyon dur” iyon / Anyon 6 — dur” iyon 
(Cation “is an ” ion / Anion “is an ’’ ion) 


Katyon arti yiiklii iyondur. / anyon eksi yiiklii 
iyondur. 

(Cation is a positively charged ion / Anion is 
a negatively charged ion) 


Fatma 


Nehir 


Katyon dur” iyon / Anyon dur” iyon. 
(Cation “is an ” ion / Anion “is an ” ion) 


Katyon pozitif yiiklii iyondur / Anyon negatif 
yiiklii iyondur 

(Cation is a positively charged ion / Anion is 
a negatively charged ion) 


Katyon arti yiiklii iyondur. / Anyon eksi 
iyon “olabilir” katyon / iyon “olabilir” anyon yuklii iyondur 

(Ion can be a cation /Ion can be an anion) (Cation is a positively charged ion / Anion is 

a negatively charged ion) 


Data related to Pre-Service Teachers’ Opinions 

Forty-two codes were determined as a result of the opinion content analysis, 
and they were gathered under 6 themes: clarity, understandability, assessment tool 
usage, forming relations, usability, and compatibility with Turkish. The operational 
descriptions of the themes are tabulated in Table 3, and the number of participants 
who stated opinions is listed in Table 4. 


Table 3 



Operational Description of the Themes 


Themes 

Number of codes 
in the theme 

Operational description of the themes 

Clarity 

10 

Includes phrases related to clarity of relations. 

Understandability 

15 

Includes phrases related to relational and/or others’ un¬ 
derstandability. 

Assessment tool usage 

2 

Includes phrases about when maps are used as an assess¬ 
ment tool 

Forming relations 

7 

Includes phrases used in forming relations 

Usability 

4 

Includes phrases related to the usability of the system 

Compatibility with Turkish 

4 

Includes phrases related to Turkish syntax structure 
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Table 4 


Number of Participants who Stated Opinions 

Themes 

Number of Codes in the 
Theme 

Number of participants who 
stated opinions in the theme 

Percentage 

Clarity 

10 

9 

45% 

Understandability 

15 

14 

70% 

Assessment tool usage 

2 

2 

10% 

Forming relations 

7 

5 

25% 

Usability 

4 

4 

20% 

Compatibility with Turkish 

4 

2 

10% 


When examining the opinions listed under clarity, 45% of participants were seen 
to state the relations on the maps to be much more explanatory and grammatically 
compatible when using the indexing method. Participants were observed to fail at 
phrasing the relationship when they used the classical Novakian approach. Some of 
the opinions given below show the differences between the Novakian and indexing 
systems in terms of clarity: 

[Indexing method] makes establishing the relation much easier and clearer. (Esma) 

The phrases [in the Novakian maps] remained in its simplest form as plays, does, 
etc. We are not allowed to use suffixes. That’s why it was difficult to fully phrase the 
relation. (Deniz) 

[Indexing method] is much more useful as it provides more meaningful relational 
clauses. (Pelin) 

I think phrases [in the Novakian method] are much more limited. (Melek) 

Sixty-five percent of participants who stated opinions under the theme of 
understandability of the resultant relational clauses agreed that the clauses established 
through the indexing method were much more superior in terms of syntax order and 
informational content. According to the participants, the indexing method promotes 
the readability of relational clauses. Some of the pre-service teachers’ opinions about 
this topic clarify this point: 

Indexing is important because when we write phrases directly it becomes hard to read. 
Therefore, it makes reading and understanding much easier. (Mine) 

I think that using indexing obviates confusion. (Esma) 

I think that indexing is very important because without it, the resulting clause becomes 
very complicated. (Fatma) 

It is very easy to establish meaningful clauses with the indexing method. Clauses 
become much more meaningful and clear. (Zeynep) 

Only one pre-service teacher stated that the numbers used in the indexing method 
made it complicated. 
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Concept maps may be more informative with the indexing method, but they are also 
more complicated. (Beyhan) 

This was the only negative opinion about the indexing method’s understandability. 

Participants’ statement about the effect of using different concept mapping 
methods in preparing concept maps as an assessment tool are placed under the theme 
of assessment tool usage. There are two opinions expressed under this heading: One 
stated that the difference was minimal, but the second emphasized that the indexing 
method was much more effective. The opinions of the pre-service teachers about the 
use of concept maps as an assessment tool are as follows: 

I don’t think there’s any difference between the two systems. The indexing system is 
more convenient due to ease of phrasing. (Beyhan) 

The indexing model is much more convenient as an assessment tool due to its phrasal 
diversity. (Deniz) 

Pre-service teachers’ opinions about forming relations are generally positive, 
stating that clauses resulting from the indexing method are much more informative 
and grammatically correct. They also stated that when using the indexing method, 
expressing and writing phrases about what they were trying to explain was much 
easier. Here are some of the participants’ opinions about this topic: 

I felt a lot more comfortable when I was preparing the maps according to the indexing 
technique because it was much easier to express what I wanted to explain. I think the 
indexing method is more practical regarding ease of expression. (Sinem) 

It was very difficult to write phrases in the Novakian system. It took a very long time 
to form a relation. (Zeynep) 

I think Novakian phrases are too limited. It’s much easier to define phrases in the 
indexing system. (Melek) 

The indexed approach is much easier for establishing appropriate clauses. It’s quite 
difficult to do the same using the Novakian method. (Nehir) 

The opinions of the pre-service teachers about the Novakian and indexing methods’ 
usability were completely in favor of the latter. Here are some opinions about the 
usability of the systems: 

Indexed concept maps are more useful because they are easier to understand. (Melek) 

I could easily establish phrases using the index method. I think the indexing system is 
much more useful. (Sinem) 

Indexed maps are a lot more useful because they provide much more meaningful 
clauses. (Pelin) 

It is more practical to use the indexing method. Indexing relation as 1 and 2 eliminates 
the possibility of reading incorrectly, (ipek) 
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According to the participants, the Novakian method is not compatible with Turkish 
because it is not possible to include suffixes, the most important feature of Turkish. 
The resultant clauses are generally meaningless and grammatically distorted. 
However, there is no limitation in terms of suffixes with the indexing method, which 
results in meaningful clauses in line with the grammatical structure of Turkish. Here 
are the opinions of some pre-service teachers: 

The indexed format allows for clauses that are compatible with the Turkish language. 

(Mehtap) 

ft was highly difficult to write phrases without suffixes. (Canan) 

Discussion 

Most of the literature related to the problems emerging while preparing concept 
maps in Turkish focus upon the problems associated with writing phrases (Didi§ et al., 
2014; inge?, 2009; Lee, 1999; Park & Kim, 2008; Saglam, 2009; Sozbilir & Neac§u, 
2014; Turan & Elcmekci, 2011; Unlii et al., 2006). The structural differences between 
Turkish and English cause distortions in the resultant clauses and make it difficult to 
write propositions. This causes Novakian concept maps to deviate from their primary 
use of elucidating relations between concepts, and the conceptual structure about a 
certain topic resurfaces. This study is related to the development of an entirely new 
method, the indexing method, to remedy all problems involved in preparing Novakian 
maps in Turkish. Indexed concept maps were found to be richer in informational 
content. The Novakian system is prepared by using one- or two-word phrases in their 
simplest form. However, the indexing approach paves the way for including suffixes. 
Because the order of words in reading is not clear with the classical Novakian method, 
the difference sometimes contains gross errors or misconceptions. The indexing 
method is found to remedy most of the problems by improving readability and 
informational content. When dwelling upon the participants’ statements regarding 
the indexing system’s effect on understandability of Novakian maps, they are seen 
to completely agree that the indexing system greatly increased the final statement’s 
understandability. The peer readings show that Novakian relations are incorrectly read 
most of the time, and using the indexing method obviates most of these problems. 
The problems observed in Novakian maps were in accordance with those reported by 
Bagci Kilip (2003) and Unlii et al. (2006). Bagci Kiln; (2003) reported the structural 
difference between Turkish and English to be mainly responsible for the problems of 
concept maps in Turkish, fn our study, the participants stated having serious problems 
preparing concept maps in Turkish and claimed the indexing method to be much better 
suited for this purpose. Bagci Kiln; (2003) stated the failure of adding suffixes at the 
end of the verb to inevitably cause problems in relational clauses and propositions in 
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Novakian maps would be meaningless due to the two languages’ difference in syntax 
structure. Unlii et al. (2006) reported in their study that they had so many problems 
due to the syntaxial differences between Turkish and English; they stated that the 
expressions between concepts describing their relationship were often incompatible 
with Turkish grammatical structure. They claimed that this syntaxial distortion could 
well result in misconceptions. Participants’ written opinions revealed that they had 
difficulties establishing clauses and were forced to establish relations using short 
phrases with little expressive power. Investigations of the participants’ concept maps 
clearly showed this (see Figure 11 and Table 2). Additionally, data collected from 
the opinions listed under clarity and forming relations support this situation. The 
participants were reported to have had great difficulty establishing relational clauses 
between two concepts using a few simple words in the Novakian tables because of 
the restrictions imposed by Novak’s method, even if they understood the relation. 
The numbering method developed by Saglam (2009), where phrases are replaced 
with straight clauses, eliminated the poor grammatical structure inherent in the 
Novakian method. However, because the maps and relationships are on different 
pages, following the whole process becomes a huge problem (Saglam, 2009). This 
situation also clearly violates the basic feature of concept maps: “concept maps show 
students’ informational structure in an integrated fashion” (Novak & Musonda, 1991). 
In this context, the indexing system appears to be superior. The fact that the phrases 
are written on the relationship line is the biggest advantage of the indexing approach. 
This system is thought to enable students to establish concept maps much more 
easily, as well as provide teachers with the opportunity to more realistically assess the 
relations that students establish. The problem of complicating maps by writing whole 
clauses on the relational lines, as stated by Saglam (2009) in his numbering system, 
exists in the indexing system as well. However, writing clauses on the relational lines 
are very important if one is to take the main target of concept mapping into account: 
visualizing students’ concepts. 

When looking at the pre-service teachers ’ opinions, almost all of them stated positive 
opinions about the indexing method. Participants (65%) stated that the indexing 
method was much more compatible and understandable with Turkish compared to 
the Novakian method. Only one participant expressed concern that the indexing 
system would complicate things. However, the rest of the participants agreed that 
phrasing maps and establishing relations is much easier using the indexing method. 
They thought the resulting relations were much more understandable compared to 
the classical Novakian method. This data complies with that from the study of Didi§ 
et al. (2014), which investigated students’ opinions about concept maps. As a result 
of this study, concept maps should be clearly understandable, appealing, and rich in 
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informational content. In summary, concept maps prepared in accordance with the 
indexing method are much more clear and understandable. 

When the Novakian and indexing methods are evaluated using the Novakian scoring 
method (Novak & Gowin, 1984), both mapping systems score the same points because 
every proposition is given 1 point regardless of its quality. However, if one considers 
that the primary purpose of concept maps is to illuminate students’ conceptual structures, 
indexed concept maps are a better alternative. Take the concept maps prepared by pre¬ 
service teacher “Melek” as an example: “Aneutron is located in the nucleus” is a correct 
proposition and is graded with one point. However, the proposition “Neutral particles 
in the nucleus are called neutrons” is also graded with one point. A vast difference 
is reflected in regard to informational content in these two propositions. Lopez et al. 
(2011) proposed a four-tier grading system: 0 = wrong or scientifically inappropriate, 
1 = partly wrong, 2 = correct but cognitively weak, and 3 = scientifically correct and 
fully clear. The correct but cognitively weak category fits well the small phrases used 
in Novakian-type maps. On the other hand, the much clearer expressions that are used 
with the indexing approach fall into the final category. Therefore, most small phrases 
in Novakian maps score two points. Although Novakian grading does not differentiate 
between the Novakian and indexed approaches, detailed grading clearly indicates the 
superiority of the indexing system as an assessment tool over the traditional Novakian 
approach when the application language is Turkish. Although concept maps are 
graded qualitatively, they are used as a quantitative tool in assessment for enlightening 
students’ misconceptions and conceptual structures (Koca & Sen, 2004). In this type 
of application, participants’ relational phrases are of great importance. That is why the 
indexing approach is much more superior for elucidating students’ conceptual structures. 
Two participants compared the Novakian approach and the indexing method in terms 
of being an assessment tool. One of them stated the difference to be minimal, and the 
second claimed the indexing method to be much more superior. 

Concept maps are two-dimensional schemes for determining how students relate 
concepts and define their prior knowledge and alternative concepts (Novak & Gowin, 
1984; Ruiz-Primo & Shavelson, 1996). This study showed that some of the relational 
phrases in Novakian maps that could be regarded as misconceptions give perfectly 
sound and conceptually correct relational clauses in the indexing system, which 
indicates that the incorrect phrasing did not result from a misconception. Concept 
maps are tools that can be employed to elucidate misconceptions (Aykutlu & §en, 
2012; £ildir & §en, 2006; Djanettea & Fouadb, 2014; Karamustafaoglu et al., 2002; 
Ta§9i, 2015). That is why it is recommended to employ the indexing method when 
concept maps are used as a tool for defining misconceptions in Turkish due to its 
greater power of expression. 
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Limitations and Proposals 

All data collection tools being in written format and the lack of individual participant 
interviews are the biggest limitations of the study. Also, the study being limited 
to only 20 participants is another important limitation that decreases the external 
validity of the study. The decreased quota for chemistry teachers made finding an 
adequate number of samples quite difficult. Future studies are recommended to be 
performed with larger sampling groups, including participants from other disciplines 
if necessary. Additionally, study topics are recommended to be enumerated, and both 
written and oral data collection methods should be employed. The indexing method 
should be introduced to teachers prior to the application. Teachers who use Novakian 
concept maps as an assessment tool in Turkish in their classes should be urged to use 
the new indexing system. The need to write relational phrases may complicate the 
process but is absolutely necessary for integrating the process. In order to obviate 
this problem, maps could be drawn on larger-sized paper and the number of concepts 
could be restricted by dividing the topic. 
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Appendix 

Operational Description of the Themes Used in Analyzing Participants ’ Novakian 
and Indexed Concept Maps 


Category Name 

Operational Description 

Correct phrase 

Proper and compatible with the scientific model’s proposition written between two 
concepts. 

Wrong phrase 

Wrong or misconceived proposition written between two concepts. 

Concept error 

Incorrect proposition because it connects to the wrong concept. 

Incompatible with the 
scientific model 

The proposition is not suitable to the current scientific model but is appropriate to 
an out-of-date model. 

Phrase with excessive 
information 

The propositions between the same two concepts in different mapping types 
contain an excess of information. 
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